A New Approach to Automatic Summarization by Using Latent Dirichlet Allocation in Conditional Random Field

A New Approach to Automatic Summarization by Using Latent Dirichlet Allocation in Conditional Random Field Xiaofeng Wu, Chengqing Zong (National Lab of Pattern Recognition, Institute of Automation, CAS, Beijing 100190, China) Abustract: In recent years, Latent Dirichlet Allocation(LDA) has been used more and more in Document Clustering, Classification, Segmentation, and some one has used it in query based multi-document summarization, which is an unsupervised approach. LDA is recognized by its great power in modeling a document in a semantic way. In this paper we propose a new approach to extractive-based, supervised single document summarization, which called LDA based CRF (Conditional Random Field) Automatic Summarization (LCAS), by adding of Latent Dirichlet Allocation of the document as new features into a CRF summarization system. We study the power of LDA, and analyze its different effects by changing the number of topics. Our experiments show that, by adding LDA features, the result of traditional CRF summarization system can be impressively increased.